{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ljRgQzDLZrv6"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/langchain/rag-chatbot.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/generation/langchain/rag-chatbot.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "85Xhv9IcZrv7"
      },
      "source": [
        "# Building RAG Chatbots with LangChain"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yHSntiUHZrv7"
      },
      "source": [
        "In this example, we'll work on building an AI chatbot from start-to-finish. We will be using LangChain, OpenAI, and Pinecone vector DB, to build a chatbot capable of learning from the external world using **R**etrieval **A**ugmented **G**eneration (RAG).\n",
        "\n",
        "We will be using a dataset sourced from the Deepseek R1 ArXiv paper to help our chatbot answer questions about the latest and greatest in the world of AI.\n",
        "\n",
        "By the end of the example we'll have a functioning chatbot and RAG pipeline that can hold a conversation and provide informative responses based on a knowledge base.\n",
        "\n",
        "### Before you begin\n",
        "\n",
        "You'll need to get an [OpenAI API key](https://platform.openai.com/account/api-keys) and [Pinecone API key](https://app.pinecone.io)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UIWCb3LVZrv7"
      },
      "source": [
        "### Prerequisites"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NUfghzFtZrv8"
      },
      "source": [
        "Before we start building our chatbot, we need to install some Python libraries. Here's a brief overview of what each library does:\n",
        "\n",
        "- **langchain**: This is a library for GenAI. We'll use it to chain together different language models and components for our chatbot.\n",
        "- **openai**: This is the official OpenAI Python client. We'll use it to interact with the OpenAI API and generate responses for our chatbot.\n",
        "- **datasets**: This library provides a vast array of datasets for machine learning. We'll use it to load our knowledge base for the chatbot.\n",
        "- **pinecone-client**: This is the official Pinecone Python client. We'll use it to interact with the Pinecone API and store our chatbot's knowledge base in a vector database.\n",
        "\n",
        "You can install these libraries using pip like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pIVObjB3Zrv8",
        "outputId": "cca9b4ba-6383-41e4-b64d-7510703cc9d8"
      },
      "outputs": [],
      "source": [
        "!pip install -qU \\\n",
        "    langchain==0.3.23 \\\n",
        "    langchain-community==0.3.21 \\\n",
        "    langchain-pinecone==0.2.5 \\\n",
        "    langchain-openai==0.3.12 \\\n",
        "    datasets==3.5.0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fqiSJjhmZrv9"
      },
      "source": [
        "### Building a Chatbot (no RAG)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e_qCi2M0Zrv9"
      },
      "source": [
        "We will be relying heavily on the LangChain library to bring together the different components needed for our chatbot. To begin, we'll create a simple chatbot without any retrieval augmentation. We do this by initializing a `ChatOpenAI` object. For this we do need an [OpenAI API key](https://platform.openai.com/account/api-keys)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "hJXUocdtZrv9"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from langchain_openai import ChatOpenAI\n",
        "from getpass import getpass\n",
        "\n",
        "os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\") or getpass(\n",
        "    \"Enter your OpenAI API key: \"\n",
        ")\n",
        "\n",
        "chat = ChatOpenAI(\n",
        "    openai_api_key=os.environ[\"OPENAI_API_KEY\"],\n",
        "    model='gpt-4o-mini'\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DYaKQneDZrv9"
      },
      "source": [
        "Chats with OpenAI's `gpt-3.5-turbo` and `gpt-4` chat models are typically structured (in plain text) like this:\n",
        "\n",
        "```\n",
        "System: You are a helpful assistant.\n",
        "\n",
        "User: Hi AI, how are you today?\n",
        "\n",
        "Assistant: I'm great thank you. How can I help you?\n",
        "\n",
        "User: I'd like to understand string theory.\n",
        "\n",
        "Assistant:\n",
        "```\n",
        "\n",
        "The final `\"Assistant:\"` without a response is what would prompt the model to continue the conversation. In the official OpenAI `ChatCompletion` endpoint these would be passed to the model in a format like:\n",
        "\n",
        "```python\n",
        "[\n",
        "    {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
        "    {\"role\": \"user\", \"content\": \"Hi AI, how are you today?\"},\n",
        "    {\"role\": \"assistant\", \"content\": \"I'm great thank you. How can I help you?\"}\n",
        "    {\"role\": \"user\", \"content\": \"I'd like to understand string theory.\"}\n",
        "]\n",
        "```\n",
        "\n",
        "In LangChain there is a slightly different format. We use three _message_ objects like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "cbfvMP4MZrv-"
      },
      "outputs": [],
      "source": [
        "from langchain.schema import (\n",
        "    SystemMessage,\n",
        "    HumanMessage,\n",
        "    AIMessage\n",
        ")\n",
        "\n",
        "messages = [\n",
        "    SystemMessage(content=\"You are a helpful assistant.\"),\n",
        "    HumanMessage(content=\"Hi AI, how are you today?\"),\n",
        "    AIMessage(content=\"I'm great thank you. How can I help you?\"),\n",
        "    HumanMessage(content=\"I'd like to understand string theory.\")\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VQNiV_YAZrv-"
      },
      "source": [
        "The format is very similar, we're just swapped the role of `\"user\"` for `HumanMessage`, and the role of `\"assistant\"` for `AIMessage`.\n",
        "\n",
        "We generate the next response from the AI by passing these messages to the `ChatOpenAI` object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jfsMnGrsZrv-",
        "outputId": "67cfebd5-3abb-42fb-bc02-0549964265e9"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "<ipython-input-4-04401e4b135d>:1: LangChainDeprecationWarning: The method `BaseChatModel.__call__` was deprecated in langchain-core 0.1.7 and will be removed in 1.0. Use :meth:`~invoke` instead.\n",
            "  res = chat(messages)\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "AIMessage(content='String theory is a theoretical framework in physics that attempts to unify all fundamental forces of nature, including gravity, quantum mechanics, electromagnetism, and the strong and weak nuclear forces. Here are some key concepts to help you understand string theory better:\\n\\n1. **Basic Idea**: At its core, string theory posits that the fundamental building blocks of the universe are not point-like particles (as in traditional particle physics) but rather one-dimensional \"strings.\" These strings can vibrate at different frequencies, and the different vibrational modes correspond to different particles.\\n\\n2. **Dimensions**: String theory requires additional spatial dimensions beyond the familiar three dimensions of space and one dimension of time. In many versions of string theory, there are 10 or 11 total dimensions. The extra dimensions are typically compactified, meaning they are curled up and not directly observable at human scales.\\n\\n3. **Types of Strings**: There are open strings (which have two ends) and closed strings (which form loops). Different types of strings can lead to different kinds of particles and fundamental interactions.\\n\\n4. **Supersymmetry**: Many string theories incorporate the concept of supersymmetry, which posits a symmetry between bosons (force-carrying particles) and fermions (matter particles). This idea helps to address certain theoretical issues and is an essential feature of many string theory models.\\n\\n5. **M-Theory**: M-theory is an extension of string theory that includes different string theories as its limits. It suggests that in higher-dimensional spacetime, strings can also take the form of two-dimensional surfaces called membranes (or \"branes\").\\n\\n6. **Quantum Gravity**: One of the significant motivations for string theory is to provide a consistent theory of quantum gravity—an understanding of gravity at the quantum level, where the traditional theories of general relativity and quantum mechanics cannot be reconciled.\\n\\n7. **Challenges and Criticism**: String theory is still largely theoretical and has yet to make definitive predictions that can be tested through experimentation. One major criticism is that it lacks empirical evidence. The theory also faces challenges in generating predictions that can be falsified, which is an important criterion in scientific theories.\\n\\n8. **Current Research**: Research in string theory is ongoing, and it continues to evolve. The mathematical frameworks are often complex, and physicists are exploring various approaches to harmonize string theory with observable physics.\\n\\nThis summary provides a basic overview of string theory, but it is a deep and complex subject with much ongoing research. If you have specific questions or topics related to string theory that you\\'d like to explore further, feel free to ask!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 531, 'prompt_tokens': 51, 'total_tokens': 582, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_44added55e', 'id': 'chatcmpl-BMcU2vkSicFqKMEYQLcB2w9iXiWQz', 'finish_reason': 'stop', 'logprobs': None}, id='run-6501c5fb-6f1f-4d87-811a-08584faf072e-0', usage_metadata={'input_tokens': 51, 'output_tokens': 531, 'total_tokens': 582, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "res = chat(messages)\n",
        "res"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9xhDvRR_Zrv-"
      },
      "source": [
        "In response we get another AI message object. We can print it more clearly like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O056MNkIZrv-",
        "outputId": "c7795037-1b51-4453-9bf9-9d428f460a50"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "String theory is a theoretical framework in physics that attempts to unify all fundamental forces of nature, including gravity, quantum mechanics, electromagnetism, and the strong and weak nuclear forces. Here are some key concepts to help you understand string theory better:\n",
            "\n",
            "1. **Basic Idea**: At its core, string theory posits that the fundamental building blocks of the universe are not point-like particles (as in traditional particle physics) but rather one-dimensional \"strings.\" These strings can vibrate at different frequencies, and the different vibrational modes correspond to different particles.\n",
            "\n",
            "2. **Dimensions**: String theory requires additional spatial dimensions beyond the familiar three dimensions of space and one dimension of time. In many versions of string theory, there are 10 or 11 total dimensions. The extra dimensions are typically compactified, meaning they are curled up and not directly observable at human scales.\n",
            "\n",
            "3. **Types of Strings**: There are open strings (which have two ends) and closed strings (which form loops). Different types of strings can lead to different kinds of particles and fundamental interactions.\n",
            "\n",
            "4. **Supersymmetry**: Many string theories incorporate the concept of supersymmetry, which posits a symmetry between bosons (force-carrying particles) and fermions (matter particles). This idea helps to address certain theoretical issues and is an essential feature of many string theory models.\n",
            "\n",
            "5. **M-Theory**: M-theory is an extension of string theory that includes different string theories as its limits. It suggests that in higher-dimensional spacetime, strings can also take the form of two-dimensional surfaces called membranes (or \"branes\").\n",
            "\n",
            "6. **Quantum Gravity**: One of the significant motivations for string theory is to provide a consistent theory of quantum gravity—an understanding of gravity at the quantum level, where the traditional theories of general relativity and quantum mechanics cannot be reconciled.\n",
            "\n",
            "7. **Challenges and Criticism**: String theory is still largely theoretical and has yet to make definitive predictions that can be tested through experimentation. One major criticism is that it lacks empirical evidence. The theory also faces challenges in generating predictions that can be falsified, which is an important criterion in scientific theories.\n",
            "\n",
            "8. **Current Research**: Research in string theory is ongoing, and it continues to evolve. The mathematical frameworks are often complex, and physicists are exploring various approaches to harmonize string theory with observable physics.\n",
            "\n",
            "This summary provides a basic overview of string theory, but it is a deep and complex subject with much ongoing research. If you have specific questions or topics related to string theory that you'd like to explore further, feel free to ask!\n"
          ]
        }
      ],
      "source": [
        "print(res.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wTTa9OtsZrv-"
      },
      "source": [
        "Because `res` is just another `AIMessage` object, we can append it to `messages`, add another `HumanMessage`, and generate the next response in the conversation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wrMsBypWZrv-",
        "outputId": "9da99810-1dbe-4677-cc22-dd340311a22a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Physicists believe string theory has the potential to produce a \"unified theory\" — often called a theory of everything — for several reasons:\n",
            "\n",
            "1. **Inclusion of Different Forces**: String theory naturally incorporates all four fundamental forces of nature (gravity, electromagnetism, the weak nuclear force, and the strong nuclear force) into one framework. Traditional particle physics, particularly the Standard Model, successfully describes electromagnetic and weak interactions but struggles to include gravity. String theory, with its additional dimensions and fundamental strings, can account for gravity by treating it as a manifestation of strings vibrating in a higher-dimensional space.\n",
            "\n",
            "2. **Quantum Gravity**: One of the major successes of string theory is that it provides a consistent description of gravity at the quantum level. In contrast, attempts to quantize gravity using conventional methods lead to inconsistencies and infinities. String theory's framework manages to avoid these problems, suggesting a coherent way to merge general relativity (which describes gravity on large scales) with quantum mechanics (which governs the behavior of particles on very small scales).\n",
            "\n",
            "3. **Vibrational Modes Correspond to Particles**: In string theory, different particles are viewed as different vibrational states of strings. This means that instead of treating particles as distinct entities, they can be understood as manifestations of the same fundamental object (the string) vibrating in different ways. This unified perspective is a powerful conceptual improvement over treating particles as fundamentally separate.\n",
            "\n",
            "4. **Supersymmetry**: Many string theory models incorporate supersymmetry, which posits a relationship between particles with different spins. This relationship can help to unify the forces because it allows particles and forces to transform into one another under certain conditions, enhancing the mathematical harmony of the theory.\n",
            "\n",
            "5. **Higher Dimensions**: String theory elegantly includes additional dimensions beyond the familiar four. The existence of these extra dimensions allows for a more complex and unified description of the interactions between fundamental forces. The behavior of strings in these extra dimensions can produce different physical phenomena, potentially explaining a wide range of observations.\n",
            "\n",
            "6. **Dualities**: String theory exhibits various dualities, indicating that different string theories can be seen as manifestations of the same underlying physics. This suggests that there is a deeper level of unity among different physical theories, hinting at a more profound underlying structure to the universe.\n",
            "\n",
            "7. **Mathematical Consistency**: String theory has a high degree of mathematical structure and has led to new insights in many areas of mathematics and physics. This mathematical rigor suggests that it may be on the right track toward providing a coherent description of the universe.\n",
            "\n",
            "8. **Ongoing Research and Development**: Physicists continue to explore string theory through both theoretical developments and potential experimental tests, such as searches for supersymmetric particles or gravitational waves that might provide indirect evidence for string-theoretic predictions.\n",
            "\n",
            "While string theory shows great promise in providing a unified theory of physics, it remains a work in progress. There are many open questions, and rigorous experimental validation is still needed to confirm its predictions and practical applicability. Researchers are continuing to work on the theory, looking for ways to bridge the gap between abstract theoretical predictions and observable phenomena.\n"
          ]
        }
      ],
      "source": [
        "# add latest AI response to messages\n",
        "messages.append(res)\n",
        "\n",
        "# now create a new user prompt\n",
        "prompt = HumanMessage(\n",
        "    content=\"Why do physicists believe it can produce a 'unified theory'?\"\n",
        ")\n",
        "# add to messages\n",
        "messages.append(prompt)\n",
        "\n",
        "# send to chat-gpt\n",
        "res = chat(messages)\n",
        "\n",
        "print(res.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2S3qX_2kZrv_"
      },
      "source": [
        "### Dealing with Hallucinations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4IrRaunDZrv_"
      },
      "source": [
        "We have our chatbot, but as mentioned — the knowledge of LLMs can be limited. The reason for this is that LLMs learn all they know during training. An LLM essentially compresses the \"world\" as seen in the training data into the internal parameters of the model. We call this knowledge the _parametric knowledge_ of the model.\n",
        "\n",
        "By default, LLMs have no access to the external world.\n",
        "\n",
        "The result of this is very clear when we ask LLMs about more recent information, like about Deepseek R1."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "CviCeCnNZrv_"
      },
      "outputs": [],
      "source": [
        "# add latest AI response to messages\n",
        "messages.append(res)\n",
        "\n",
        "# now create a new user prompt\n",
        "prompt = HumanMessage(\n",
        "    content=\"What is so special about Deepseek R1?\"\n",
        ")\n",
        "# add to messages\n",
        "messages.append(prompt)\n",
        "\n",
        "# send to OpenAI\n",
        "res = chat(messages)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z8HKTsClZrv_",
        "outputId": "19dc14cb-a794-467a-d84b-1ac7179e4c6a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "As of my last knowledge update in October 2023, the Deepseek R1 is a notable advancement in the field of underwater exploration and robotics. Here are several features and aspects that may make the Deepseek R1 special:\n",
            "\n",
            "1. **Versatile Underwater Capabilities**: The Deepseek R1 is designed to operate effectively in various underwater environments, making it valuable for research, exploration, inspection, and surveillance tasks in both freshwater and saltwater.\n",
            "\n",
            "2. **High-Quality Imaging**: Often equipped with advanced imaging technology, the Deepseek R1 may offer high-resolution cameras, sonar systems, or even 3D imaging capabilities, allowing for detailed exploration and analysis of underwater environments.\n",
            "\n",
            "3. **Autonomous Operation**: Many modern underwater drones, including the Deepseek R1, are capable of autonomous navigation. This means they can map, explore, and even conduct tasks without constant human control, which is vital for missions in challenging or hazardous underwater conditions.\n",
            "\n",
            "4. **User-Friendly Interface**: The R1 typically features an intuitive user interface that makes it accessible for both experienced operators and those new to underwater robotics, facilitating ease of use in various applications, from research to recreational exploration.\n",
            "\n",
            "5. **Modular Design**: Many advanced underwater drones are designed with modular components, allowing users to customize the vehicle for specific missions by attaching different sensors, tools, or payloads according to the requirements.\n",
            "\n",
            "6. **Data Collection and Analysis**: The Deepseek R1 often supports various data collection methods, allowing it to gather crucial information from underwater environments, which can be later analyzed for research or practical applications.\n",
            "\n",
            "7. **Durability and Reliability**: Given the demanding conditions of underwater environments, the R1 is typically constructed to be robust and reliable, capable of withstanding pressures and protecting its internal components from water and corrosion.\n",
            "\n",
            "8. **Potential for Environmental Monitoring**: The capabilities of the Deepseek R1 make it an excellent tool for monitoring the health of aquatic ecosystems, including assessing water quality, monitoring marine life, and aiding in conservation efforts.\n",
            "\n",
            "9. **Support for Wide Applications**: The R1 can be utilized in various sectors, including marine biology, underwater archaeology, oil and gas inspection, and search-and-rescue operations, making it a versatile asset across multiple fields.\n",
            "\n",
            "10. **Innovative Technology**: If applicable, the R1 may incorporate cutting-edge technology such as AI-based navigation, obstacle avoidance mechanisms, or real-time data transmission capabilities for enhanced operational efficiency.\n",
            "\n",
            "Please verify with current sources or the manufacturer's latest updates for specific details regarding the Deepseek R1, as advancements may have occurred since my last knowledge update. If you have specific aspects of the Deepseek R1 you are interested in, feel free to ask!\n"
          ]
        }
      ],
      "source": [
        "print(res.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j9g7us9cZrv_"
      },
      "source": [
        "Our chatbot can no longer help us, it doesn't contain the information we need to answer the question. It was very clear from this answer that the LLM doesn't know the informaiton, but sometimes an LLM may respond like it _does_ know the answer — and this can be very hard to detect."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MV13O32_Zrv_"
      },
      "source": [
        "There is another way of feeding knowledge into LLMs. It is called _source knowledge_ and it refers to any information fed into the LLM via the prompt. We can try that with the Deepseek question. We can take the paper abstract from the [Deepseek R1 paper](https://arxiv.org/abs/2501.12948)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "7nJOOE5JZrv_"
      },
      "outputs": [],
      "source": [
        "source_knowledge = (\n",
        "    \"We introduce our first-generation reasoning models, DeepSeek-R1-Zero and \"\n",
        "    \"DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale \"\n",
        "    \"reinforcement learning (RL) without supervised fine-tuning (SFT) as a \"\n",
        "    \"preliminary step, demonstrates remarkable reasoning capabilities. Through \"\n",
        "    \"RL, DeepSeek-R1-Zero naturally emerges with numerous powerful and \"\n",
        "    \"intriguing reasoning behaviors. However, it encounters challenges such as \"\n",
        "    \"poor readability, and language mixing. To address these issues and \"\n",
        "    \"further enhance reasoning performance, we introduce DeepSeek-R1, which \"\n",
        "    \"incorporates multi-stage training and cold-start data before RL. \"\n",
        "    \"DeepSeek-R1 achieves performance comparable to OpenAI-o1-1217 on \"\n",
        "    \"reasoning tasks. To support the research community, we open-source \"\n",
        "    \"DeepSeek-R1-Zero, DeepSeek-R1, and six dense models (1.5B, 7B, 8B, 14B, \"\n",
        "    \"32B, 70B) distilled from DeepSeek-R1 based on Qwen and Llama.\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pemRysu3ZrwA"
      },
      "source": [
        "We can feed this additional knowledge into our prompt with some instructions telling the LLM how we'd like it to use this information alongside our original query."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "YpkK0mBPZrwA"
      },
      "outputs": [],
      "source": [
        "query = \"What is so special about Deepseek R1?\"\n",
        "\n",
        "augmented_prompt = f\"\"\"Using the contexts below, answer the query.\n",
        "\n",
        "Contexts:\n",
        "{source_knowledge}\n",
        "\n",
        "Query: {query}\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8NMMYZfBZrwA"
      },
      "source": [
        "Now we feed this into our chatbot as we were before."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "46d1FM9kZrwA"
      },
      "outputs": [],
      "source": [
        "# create a new user prompt\n",
        "prompt = HumanMessage(\n",
        "    content=augmented_prompt\n",
        ")\n",
        "# add to messages\n",
        "messages.append(prompt)\n",
        "\n",
        "# send to OpenAI\n",
        "res = chat(messages)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0ImRZzBwZrwA",
        "outputId": "40a5f579-f8bf-4245-ea49-3696482ef3e8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "DeepSeek R1 is special due to its enhanced reasoning capabilities achieved through a multi-stage training process. Unlike its predecessor, DeepSeek-R1-Zero, which was trained solely using large-scale reinforcement learning (RL), DeepSeek R1 incorporates additional steps, including cold-start data before the RL phase. This approach effectively addresses challenges like poor readability and language mixing seen in R1-Zero, resulting in improved reasoning performance. Consequently, DeepSeek R1 demonstrates performance comparable to well-known models like OpenAI-o1-1217 on reasoning tasks, making it a significant advancement in reasoning models. Additionally, the models are open-sourced, contributing to the research community by providing access to diverse dense models distilled from DeepSeek R1.\n"
          ]
        }
      ],
      "source": [
        "print(res.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EAkRSGM8ZrwA"
      },
      "source": [
        "The quality of this answer is phenomenal. This is made possible thanks to the idea of augmented our query with external knowledge (source knowledge). There's just one problem — how do we get this information in the first place?\n",
        "\n",
        "We learned in the previous chapters about Pinecone and vector databases. Well, they can help us here too. But first, we'll need a dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BMFWYYVqZrwA"
      },
      "source": [
        "### Importing the Data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3cTG_6FZrwA"
      },
      "source": [
        "In this task, we will be importing our data. We will be using the Hugging Face Datasets library to load our data. Specifically, we will be using the `\"jamescalam/deepseek-r1-paper-chunked\"` dataset. This dataset contains the Deepseek R1 paper pre-processed into RAG-ready chunks."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 307,
          "referenced_widgets": [
            "473b5b06e3464f56bd69053b1c06a1d9",
            "127599d0ea6d4949b3b1fd86c1509975",
            "09b0d26991a2425b8dc362f6030950a1",
            "661eee650e1b47bc82d154eb38647466",
            "1709c70608f24c079e1c28d07e252136",
            "ea3aa4f8dede4c22bd90f497520b705b",
            "909a0ff5c6a84261850168466a3cdf0d",
            "6d40303ba0da41c7918563221a6ab483",
            "0949a81cae2840daae7cddc11ae446ea",
            "794556a9043a414abe7e7d7d55364e82",
            "9b9368abc4ff4863bac1ede45ce043cb",
            "93e4c42eb19d4d468e76d93b4706147b",
            "0a25b93949ae43fead77e42a6ac45b8c",
            "2a0d4bd1cf81452ea11c97f8bf131995",
            "c592daea85cc4b6fb97547efc7118ccc",
            "bde12eed8ab34b00a0e9d59324983377",
            "cada592cfa6448c18e0d1c05e2755489",
            "35a2d8bff84649668cdd18fb4cd2ff2a",
            "dc7f83db946a491a97862eb8b1a6b826",
            "7b6971cd86cd451db06884ba7c7bad60",
            "7e23a433a67643e4b560ff192a55162d",
            "9751eea65ad543cbb6f9d00fe9caf4bd",
            "6465554b020443e99972b0354c4d0ea1",
            "f4aff13cda404fc2bbe0c40dd7a01ccd",
            "841da9fbd3cc49cbaeff28372a286268",
            "060040ee47e840f18d6011575583fc08",
            "b86ce9b6a1dc4865b035110d547187ec",
            "8fbcc55569744713803cca230977bd8a",
            "d9af6a1ba93e48499455e326fd0bb3c7",
            "5de725f2204b4c709f3097d107b1dc44",
            "921cd6f44e464358b18393104f9e94f9",
            "f01e321e231b4819a0dfd4a80faeb776",
            "6faf625071d74c38bef8b44dd7576e4c"
          ]
        },
        "id": "e6YR3sE4ZrwA",
        "outputId": "a3db7b45-a9ad-4ccd-95b5-b4fe933de584"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "473b5b06e3464f56bd69053b1c06a1d9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "README.md:   0%|          | 0.00/24.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "93e4c42eb19d4d468e76d93b4706147b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "train.jsonl:   0%|          | 0.00/54.4k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6465554b020443e99972b0354c4d0ea1",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Generating train split:   0%|          | 0/76 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['doi', 'chunk-id', 'chunk', 'num_tokens', 'pages', 'source'],\n",
              "    num_rows: 76\n",
              "})"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from datasets import load_dataset\n",
        "\n",
        "dataset = load_dataset(\n",
        "    \"jamescalam/deepseek-r1-paper-chunked\",\n",
        "    split=\"train\"\n",
        ")\n",
        "\n",
        "dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2fBpWyOqZrwA",
        "outputId": "26a35f21-b9d8-48d2-947e-af14a528cc38"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'doi': '2501.12948v1',\n",
              " 'chunk-id': 1,\n",
              " 'chunk': \"uestion: If a > 1, then the sum of the real solutions of √a - √a + x = x is equal to Response: <think> To solve the equation √a – √a + x = x, let's start by squaring both . . . (√a-√a+x)² = x² ⇒ a - √a + x = x². Rearrange to isolate the inner square root term:(a – x²)² = a + x ⇒ a² – 2ax² + (x²)² = a + x ⇒ x⁴ - 2ax² - x + (a² – a) = 0\",\n",
              " 'num_tokens': 145,\n",
              " 'pages': [1],\n",
              " 'source': 'https://arxiv.org/abs/2501.12948'}"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CKuloLmWZrwA"
      },
      "source": [
        "#### Dataset Overview\n",
        "\n",
        "The dataset we are using is sourced from the Deepseek R1 ArXiv papers. Each entry in the dataset represents a \"chunk\" of text from the R1 paper.\n",
        "\n",
        "Because most **L**arge **L**anguage **M**odels (LLMs) only contain knowledge of the world as it was during training, even many of the newest LLMs cannot answer questions about Deepseek R1 — at least not without this data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8KigzfQ2ZrwA"
      },
      "source": [
        "### Task 4: Building the Knowledge Base"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vtUCoidMZrwA"
      },
      "source": [
        "We now have a dataset that can serve as our chatbot knowledge base. Our next task is to transform that dataset into the knowledge base that our chatbot can use. To do this we must use an embedding model and vector database.\n",
        "\n",
        "We begin by initializing our Pinecone client, this requires a [free API key](https://app.pinecone.io)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "yg65GnRWZrwL"
      },
      "outputs": [],
      "source": [
        "from pinecone import Pinecone\n",
        "\n",
        "# get API key at app.pinecone.io\n",
        "api_key = os.getenv(\"PINECONE_API_KEY\") or getpass(\n",
        "    \"Enter your Pinecone API key: \"\n",
        ")\n",
        "\n",
        "# initialize client\n",
        "pc = Pinecone(api_key=api_key)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WN6x5Y3vZrwL"
      },
      "source": [
        "Now we setup our index specification, this allows us to define the cloud provider and region where we want to deploy our index. You can find a list of all [available providers and regions here](https://docs.pinecone.io/troubleshooting/available-cloud-regions)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "T1LKJchlZrwL"
      },
      "outputs": [],
      "source": [
        "from pinecone import ServerlessSpec, CloudProvider, AwsRegion, Metric\n",
        "\n",
        "index_name = \"deepseek-r1-rag\"\n",
        "\n",
        "if not pc.has_index(name=index_name):\n",
        "    pc.create_index(\n",
        "        name=index_name,\n",
        "        metric=Metric.DOTPRODUCT,\n",
        "        dimension=1536,  # this aligns with text-embedding-3-small dims\n",
        "        spec=ServerlessSpec(\n",
        "            cloud=CloudProvider.AWS,\n",
        "            region=AwsRegion.US_EAST_1\n",
        "        )\n",
        "    )\n",
        "\n",
        "index = pc.Index(name=index_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AulXyZWNZrwM"
      },
      "source": [
        "Our index is now ready but it's empty. It is a vector index, so it needs vectors. As mentioned, to create these vector embeddings we will OpenAI's `text-embedding-small-3` model — we can access it via LangChain like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "VrTt2PEXZrwM"
      },
      "outputs": [],
      "source": [
        "from langchain_openai import OpenAIEmbeddings\n",
        "\n",
        "embed_model = OpenAIEmbeddings(model=\"text-embedding-3-small\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "56HHXESyZrwM"
      },
      "source": [
        "Using this model we can create embeddings like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fVb0nHXbZrwM",
        "outputId": "64feb8ff-66c0-4865-8ab4-acc7315fa940"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(2, 1536)"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "texts = [\n",
        "    'this is the first chunk of text',\n",
        "    'then another second chunk of text is here'\n",
        "]\n",
        "\n",
        "res = embed_model.embed_documents(texts)\n",
        "len(res), len(res[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "neGc1-q1ZrwM"
      },
      "source": [
        "From this we get two (aligning to our two chunks of text) 1536-dimensional embeddings.\n",
        "\n",
        "We're now ready to embed and index all our our data! We do this by looping through our dataset and embedding and inserting everything in batches."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "497c762f6671423daf61f5c34a32f0b9",
            "a8bd1ad6e0864a0bbe52ada2ccd348aa",
            "30c56a394b1d4492b06b1d0b82ceb9fe",
            "bdcdfe324e894f19832c37dd52622a1d",
            "fb4f4af2a83d4e809dc2ba57f6acaa3d",
            "969d7fa5784145e3b9477b00ea1aede2",
            "ddb14c51cecd45b19f56b92b74240a7e",
            "03317540829841a1a01ab0559acc5037",
            "22ea40fa587141c0be5df5af4e2d41fe",
            "cc967a39d69742b8925a820f8ed13839",
            "ca42d726265a4de696f9eeeab7af24ab"
          ]
        },
        "id": "n5WngivbZrwM",
        "outputId": "be5f09d6-825f-4613-f6e9-bee5ef685058"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "497c762f6671423daf61f5c34a32f0b9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from tqdm.auto import tqdm  # for progress bar\n",
        "\n",
        "data = dataset.to_pandas()  # this makes it easier to iterate over the dataset\n",
        "\n",
        "batch_size = 100\n",
        "\n",
        "for i in tqdm(range(0, len(data), batch_size)):\n",
        "    i_end = min(len(data), i+batch_size)\n",
        "    # get batch of data\n",
        "    batch = data.iloc[i:i_end]\n",
        "    # generate unique ids for each chunk\n",
        "    ids = [f\"{x['doi']}-{x['chunk-id']}\" for i, x in batch.iterrows()]\n",
        "    # get text to embed\n",
        "    texts = [x['chunk'] for _, x in batch.iterrows()]\n",
        "    # embed text\n",
        "    embeds = embed_model.embed_documents(texts)\n",
        "    # get metadata to store in Pinecone\n",
        "    metadata = [\n",
        "        {'text': x['chunk'],\n",
        "         'source': x['source']} for i, x in batch.iterrows()\n",
        "    ]\n",
        "    # add to Pinecone\n",
        "    index.upsert(vectors=zip(ids, embeds, metadata))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qj6NRUhIZrwM"
      },
      "source": [
        "We can check that the vector index has been populated using `describe_index_stats` like before:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "khkncjI9ZrwM",
        "outputId": "93203e54-e07d-4f4f-e66f-5785733b2934"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'dimension': 1536,\n",
              " 'index_fullness': 0.0,\n",
              " 'metric': 'dotproduct',\n",
              " 'namespaces': {'': {'vector_count': 76}},\n",
              " 'total_vector_count': 76,\n",
              " 'vector_type': 'dense'}"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "index.describe_index_stats()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QMEr4WMzZrwM"
      },
      "source": [
        "#### Retrieval Augmented Generation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yVldi8gVZrwM"
      },
      "source": [
        "We've built a fully-fledged knowledge base. Now it's time to link that knowledge base to our chatbot. To do that we'll be diving back into LangChain and reusing our template prompt from earlier."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mT3z5N9pZrwM"
      },
      "source": [
        "To use LangChain here we need to load the LangChain abstraction for a vector index, called a `vectorstore`. We pass in our vector `index` to initialize the object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "9Y5yhe-IZrwM"
      },
      "outputs": [],
      "source": [
        "from langchain_pinecone import PineconeVectorStore\n",
        "\n",
        "text_field = \"text\"  # the metadata field that contains our text\n",
        "\n",
        "# initialize the vector store object\n",
        "vectorstore = PineconeVectorStore(\n",
        "    index=index,\n",
        "    embedding=embed_model,\n",
        "    text_key=text_field\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2qhQO0fOZrwM"
      },
      "source": [
        "Using this `vectorstore` we can already query the index and see if we have any relevant information given our question about Llama 2."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "H_TinXSlZrwM",
        "outputId": "3827def8-989c-4fcd-fe8f-e3dc96f265c2"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[Document(id='2501.12948v1-39', metadata={'source': 'https://arxiv.org/abs/2501.12948'}, page_content='## 1.2. Summary of Evaluation Results - **Reasoning tasks:** (1) DeepSeek-R1 achieves a score of 79.8% Pass@1 on AIME 2024, slightly surpassing OpenAI-01-1217. On MATH-500, it attains an impressive score of 97.3%, performing on par with OpenAI-01-1217 and significantly outperforming other models. (2) On coding-related tasks, DeepSeek-R1 demonstrates expert level in code competition tasks, as it achieves 2,029 Elo rating on Codeforces outperforming 96.3% human participants in the competition. For engineering-related tasks, DeepSeek-R1 performs slightly better than DeepSeek-V3, which could help developers in real world tasks. - **Knowledge:** On benchmarks such as MMLU, MMLU-Pro, and GPQA Diamond, DeepSeek-R1 achieves outstanding results, significantly outperforming DeepSeek-V3 with scores of 90.8% on MMLU, 84.0% on MMLU-Pro, and 71.5% on GPQA Diamond. While its performance is slightly below that of OpenAI-01-1217 on these benchmarks, DeepSeek-R1 surpasses other closed-source models, demonstrating its competitive edge in educational tasks. On the factual benchmark SimpleQA, DeepSeek-R1 outperforms DeepSeek-V3, demonstrating its capability in handling fact-based queries. A similar trend is observed where OpenAI-01 surpasses 4o on this benchmark.'),\n",
              " Document(id='2501.12948v1-41', metadata={'source': 'https://arxiv.org/abs/2501.12948'}, page_content='However, DeepSeek-R1-Zero encounters challenges such as poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates a small amount of cold-start data and a multi-stage training pipeline. Specifically, we begin by collecting thousands of cold-start data to fine-tune the DeepSeek-V3-Base model. Following this, we perform reasoning-oriented RL like DeepSeek-R1-Zero. Upon nearing convergence in the RL process, we create new SFT data through rejection sampling on the RL checkpoint, combined with supervised data from DeepSeek-V3 in domains such as writing, factual QA, and self-cognition, and then retrain the DeepSeek-V3-Base model. After fine-tuning with the new data, the checkpoint undergoes an additional RL process, taking into account prompts from all scenarios. After these steps, we obtained a checkpoint referred to as DeepSeek-R1, which achieves performance on par with OpenAI-01-1217. We further explore distillation from DeepSeek-R1 to smaller dense models. Using Qwen2.5-32B ([Qwen, 2024b]) as the base model, direct distillation from DeepSeek-R1 outperforms applying RL on it. This demonstrates that the reasoning patterns discovered by larger base models are crucial for improving reasoning capabilities.'),\n",
              " Document(id='2501.12948v1-76', metadata={'source': 'https://arxiv.org/abs/2501.12948'}, page_content='```\\nDeepSeek-R1-Zero AIME accuracy during training\\n```\\n```')]"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query = \"What is so special about Deepseek R1?\"\n",
        "\n",
        "vectorstore.similarity_search(query, k=3)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i1Dl3bDXZrwM"
      },
      "source": [
        "We return a lot of text here and it's not that clear what we need or what is relevant. Fortunately, our LLM will be able to parse this information much faster than us. All we need is to link the output from our `vectorstore` to our `chat` chatbot. To do that we can use the same logic as we used earlier."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "EwX2BKbeZrwM"
      },
      "outputs": [],
      "source": [
        "def augment_prompt(query: str):\n",
        "    # get top 3 results from knowledge base\n",
        "    results = vectorstore.similarity_search(query, k=3)\n",
        "    # get the text from the results\n",
        "    source_knowledge = \"\\n\".join([x.page_content for x in results])\n",
        "    # feed into an augmented prompt\n",
        "    augmented_prompt = f\"\"\"Using the contexts below, answer the query.\n",
        "\n",
        "    Contexts:\n",
        "    {source_knowledge}\n",
        "\n",
        "    Query: {query}\"\"\"\n",
        "    return augmented_prompt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZTtGiHLzZrwN"
      },
      "source": [
        "Using this we produce an augmented prompt:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_KvqcqlQZrwN",
        "outputId": "33f2e4e0-1f44-4955-8708-1b1169a5b7c1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Using the contexts below, answer the query.\n",
            "\n",
            "    Contexts:\n",
            "    ## 1.2. Summary of Evaluation Results - **Reasoning tasks:** (1) DeepSeek-R1 achieves a score of 79.8% Pass@1 on AIME 2024, slightly surpassing OpenAI-01-1217. On MATH-500, it attains an impressive score of 97.3%, performing on par with OpenAI-01-1217 and significantly outperforming other models. (2) On coding-related tasks, DeepSeek-R1 demonstrates expert level in code competition tasks, as it achieves 2,029 Elo rating on Codeforces outperforming 96.3% human participants in the competition. For engineering-related tasks, DeepSeek-R1 performs slightly better than DeepSeek-V3, which could help developers in real world tasks. - **Knowledge:** On benchmarks such as MMLU, MMLU-Pro, and GPQA Diamond, DeepSeek-R1 achieves outstanding results, significantly outperforming DeepSeek-V3 with scores of 90.8% on MMLU, 84.0% on MMLU-Pro, and 71.5% on GPQA Diamond. While its performance is slightly below that of OpenAI-01-1217 on these benchmarks, DeepSeek-R1 surpasses other closed-source models, demonstrating its competitive edge in educational tasks. On the factual benchmark SimpleQA, DeepSeek-R1 outperforms DeepSeek-V3, demonstrating its capability in handling fact-based queries. A similar trend is observed where OpenAI-01 surpasses 4o on this benchmark.\n",
            "However, DeepSeek-R1-Zero encounters challenges such as poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates a small amount of cold-start data and a multi-stage training pipeline. Specifically, we begin by collecting thousands of cold-start data to fine-tune the DeepSeek-V3-Base model. Following this, we perform reasoning-oriented RL like DeepSeek-R1-Zero. Upon nearing convergence in the RL process, we create new SFT data through rejection sampling on the RL checkpoint, combined with supervised data from DeepSeek-V3 in domains such as writing, factual QA, and self-cognition, and then retrain the DeepSeek-V3-Base model. After fine-tuning with the new data, the checkpoint undergoes an additional RL process, taking into account prompts from all scenarios. After these steps, we obtained a checkpoint referred to as DeepSeek-R1, which achieves performance on par with OpenAI-01-1217. We further explore distillation from DeepSeek-R1 to smaller dense models. Using Qwen2.5-32B ([Qwen, 2024b]) as the base model, direct distillation from DeepSeek-R1 outperforms applying RL on it. This demonstrates that the reasoning patterns discovered by larger base models are crucial for improving reasoning capabilities.\n",
            "```\n",
            "DeepSeek-R1-Zero AIME accuracy during training\n",
            "```\n",
            "```\n",
            "\n",
            "    Query: What is so special about Deepseek R1?\n"
          ]
        }
      ],
      "source": [
        "print(augment_prompt(query))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s4J7wJ36ZrwN"
      },
      "source": [
        "There is still a lot of text here, so let's pass it onto our chat model to see how it performs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TgLaY2_4ZrwN",
        "outputId": "155c65a0-9ceb-48bf-8076-3c41a03b8167"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "DeepSeek-R1 is special for several reasons:\n",
            "\n",
            "1. **Enhanced Reasoning Performance**: DeepSeek-R1 incorporates a multi-stage training pipeline that leverages both reinforcement learning (RL) and cold-start data. This method addresses the limitations encountered by its predecessor, DeepSeek-R1-Zero, which had issues with readability and language mixing. As a result, DeepSeek-R1 achieves impressive performance on reasoning tasks, surpassing OpenAI-01-1217 with a score of 79.8% Pass@1 on AIME 2024 and a remarkable 97.3% on the MATH-500 benchmark.\n",
            "\n",
            "2. **Expert Performance in Coding Tasks**: DeepSeek-R1 demonstrates expert-level capabilities in coding-related tasks, achieving an Elo rating of 2,029 on Codeforces, outperforming 96.3% of human competitors. This showcases its practical utility in real-world coding challenges.\n",
            "\n",
            "3. **Strong Performance on Knowledge Benchmarks**: DeepSeek-R1 excels on various knowledge benchmarks, achieving high scores such as 90.8% on MMLU, 84.0% on MMLU-Pro, and 71.5% on GPQA Diamond. Although slightly below OpenAI-01-1217, it outperforms other closed-source models, demonstrating its competitive edge in educational and factual tasks.\n",
            "\n",
            "4. **Improved Adaptability and Fine-Tuning**: The multi-stage training process involved in developing DeepSeek-R1 includes gathering cold-start data, fine-tuning a base model, and using rejection sampling during the RL process. This careful methodology enhances its ability to handle complex tasks and improve reasoning capabilities.\n",
            "\n",
            "5. **Distillation Process**: The successful distillation of DeepSeek-R1 into smaller models demonstrates the efficacy of reasoning patterns discovered by the larger model. This capability highlights the versatility and scalability of the model for various applications.\n",
            "\n",
            "Overall, DeepSeek-R1 represents a significant advancement in reasoning models, showcasing competitive performance, expert capabilities, and a robust training approach.\n"
          ]
        }
      ],
      "source": [
        "# create a new user prompt\n",
        "prompt = HumanMessage(\n",
        "    content=augment_prompt(query)\n",
        ")\n",
        "# add to messages\n",
        "messages.append(prompt)\n",
        "\n",
        "res = chat(messages)\n",
        "\n",
        "print(res.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6trJB1oHZrwN"
      },
      "source": [
        "We can continue with another Deepseek R1:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nF1IBZGcZrwN",
        "outputId": "12ebc5af-f8fc-4f24-aed6-288e2954689b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "DeepSeek-R1 is an improvement over DeepSeek-R1-Zero in several key aspects:\n",
            "\n",
            "1. **Reasoning Performance**: DeepSeek-R1 addresses some of the challenges faced by DeepSeek-R1-Zero, such as poor readability and language mixing, enhancing its overall reasoning capabilities. This results in DeepSeek-R1 achieving a score of 79.8% Pass@1 on AIME 2024, slightly surpassing OpenAI-01-1217, while DeepSeek-R1-Zero's performance was not mentioned in this specific context.\n",
            "\n",
            "2. **Training Methodology**: DeepSeek-R1 incorporates a multi-stage training pipeline and utilizes a small amount of cold-start data before reinforcement learning (RL), which helps to refine its reasoning skills. In contrast, DeepSeek-R1-Zero was trained solely via large-scale RL without supervised fine-tuning as a preliminary step.\n",
            "\n",
            "3. **Knowledge Benchmarking**: In comparison to DeepSeek-R1-Zero, DeepSeek-R1 achieves higher scores on knowledge benchmarks such as MMLU, MMLU-Pro, and GPQA Diamond, indicating better performance in educational tasks. For example, DeepSeek-R1 scores 90.8% on MMLU, compared to DeepSeek-V3, which it significantly outperforms.\n",
            "\n",
            "4. **Coding and Engineering Tasks**: DeepSeek-R1 demonstrates an expert level in coding-related tasks, achieving a high Elo rating on Codeforces and outperforming 96.3% of human participants in competitions. This indicates a practical advantage over DeepSeek-R1-Zero, which did not achieve such explicit results.\n",
            "\n",
            "5. **Distillation and Further Development**: DeepSeek-R1 serves as a foundation for exploring the distillation into smaller dense models, showing that it has identified significant reasoning patterns during its training that can improve reasoning capabilities in these smaller models.\n",
            "\n",
            "Overall, DeepSeek-R1 improves upon the initial framework established by DeepSeek-R1-Zero by enhancing reasoning performance, incorporating advanced training techniques, and achieving superior results on various benchmarks.\n"
          ]
        }
      ],
      "source": [
        "prompt = HumanMessage(\n",
        "    content=augment_prompt(\n",
        "        \"how does deepseek r1 compare to deepseek r1 zero?\"\n",
        "    )\n",
        ")\n",
        "\n",
        "res = chat(messages + [prompt])\n",
        "print(res.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can continue asking questions about Deepseek R1, but once you're done you can delete the index to save resources:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2LVafqdfZrwN"
      },
      "outputs": [],
      "source": [
        "pc.delete_index(index_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c2ie7glFZrwN"
      },
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "redacre",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
